Public RenderOptions, FindReferencedTemplates and sandbox policy (0357 S2b, S2c) - #811

Requested by Ruslan · project thread

Problem

Task 0357 moves the Python module from private engine seams to the public API before the G1 freeze (plan docs/design/0357-python-module-public-api.md, approved 2026-11-06). #800 landed the public host object (S1a, S2a). Three seams were still private:

• the module's render entry with its own globals and Ctrl-C poll (RenderInternal, t_hostPoll);
• ReferencedTemplatesOf;
• the sandbox policy from Python module: the sandbox (SandboxedEnvironment, ImmutableSandboxedEnvironment) over an engine sandbox policy #801.

This PR makes all three public: B3, B6 and B7/D6b. The module itself switches in S3.

Resulting behaviour

S2b (B3, B6)

• jinja2::RenderOptions{ shared_ptr<const ValuesMap> globals; std::function<void()> poll; } and Template::RenderTo(out | sink, params, options).
  • The globals replace the environment's for this render and for the templates it includes, imports or extends.
  • The globals are shared rather than borrowed. A value a callable keeps stays valid after the render, as an environment global's does (LendGlobal).
  • The poll runs about every 1024 steps and may throw. The error then carries the exception in GetException().
  • A poll alone makes a render count its steps. A render that a polled one starts (a callable rendering another template) is polled too.
• Template::FindReferencedTemplates() is now public. test/referenced_templates_test.cpp now builds in shared mode.
• Bug found on the way: in a wide template every name came back nullopt. Fixed, with a test row.

S2c (B7, D6b)

• New public jinja2::ISandboxPolicy in jinja2cpp/sandbox_policy.h, held by Settings::sandboxPolicy (compared by identity).
  • NativeAttribute(const Value&, string_view) -> optional<Value> is asked wherever the engine's policy was asked before.
  • Format(...) -> optional<Value> returns nullopt by default. The engine's own str.format then reads each field's attribute through NativeAttribute, as Jinja2's SandboxedFormatter does: '{0.__class__}|'.format(s) gives |.
• The engine's interface is renamed IEngineSandboxPolicy, with one bridge (src/public_sandbox_policy.cpp), as for host objects.
• Settings::immutableContainers is the public switch for the immutable sandbox.
• New ErrorCode::SecurityError (21) replaces the extraParams[1] marker on an UndefinedError.
  • This changes the error code users see for the chat preset's refused list and dict methods: SecurityError, as in transformers' Jinja2.
• UndefinedDescription::security makes and reads a refused attribute's undefined value.
• 0427 is decided and marked done: a template keeps the policy it was loaded with, as it keeps its other settings. A test pins it.
• Python module changes are minimal: poll on the new HostPollScope, the SecurityError code mapping, the rename. Its render entry is unchanged until S3.

Validation

• Debug ctest: 8/8 passed.
• Python module (Release, boost big ints, Python 3.12): pytest 1594 passed, 17 skipped. Upstream Jinja2 suite 596/856, unchanged.
• scripts/preflight.sh --perf: all checks pass.
  • The instruction gate passes, and no row is above +0.3% locally (dec3a05). plain_text and substitute renders count the same as master; Load/plain_text is +9 instructions (+0.24%), 6 of them the Settings == comparison of the two new fields; Render/for_filter_if +0.30% (a step per item the filter skips).
  • The perf track held 859ce81 on a +14 per-render cost. Callgrind put all of it in template.cpp's copy of ~RenderContext, which called a robin_hood table destructor out of line once RenderCore<true> was compiled in the same file. TemplateImpl::RenderWithOptions now lives in src/template_render_options.cpp, and the TemplateParser constructor is forced inline.
• Verifier, 1 round: ready, no blocking findings.
  • Debug and -Werror build with full ctest.
  • Clang ASan+UBSan: the full suite passed.
  • Shared build: the new suites pass and the new symbols are exported.
  • Public headers compile alone as C++17 with -Wpedantic.
  • Expectations checked against Jinja2 3.1.6 (SandboxedEnvironment, ImmutableSandboxedEnvironment, meta.find_referenced_templates).
  • Adversarial probes: kept globals, throwing and nested polls, a throwing policy, Format results that keep lent arguments.
  • Its one non-blocking note: formatMap was never set for a public policy. The header now says format_map comes from NativeAttribute, as the Python module does it.
• Not run locally: MSVC, AppleClang, TSan (CI covers them).

Limits

• A macro or caller reaches NativeAttribute as an empty value. S3 must keep the module's stand-in object for them.
• TemplateEnvImpl::SetSandboxPolicy and SetDialect stay internal until the module switches (S3) and are deleted in S4.

Roles: verifier ×1 (1 round, ready). Red CI pushes: 1 (dec3a05: clang-tidy misc-include-cleaner in the new src/template_render_options.cpp; local preflight skipped the file because build/ had not been reconfigured).
Codex: 4 findings in 3 rounds, 4 fixed, 0 declined (e740760: a loop filter's skipped items and a backend-only limits scope now reach RenderOptions::poll, Render/for_filter_if +0.30%; da85d26: per-render globals are converted inside the render's limits scope and error boundary; 859ce81: a nested render under a backend-only scope is polled too). Both re-requests are used, so the last fix has no Codex re-review.

Generated by Claude Code